Summary

Main changes to extend the job progress tracking API with a error collection feature.

Errors are attached to the overall progress JSON object.
A new column errorCodes was added to the table which contains the codes extracted from the progress object.
This is purely to allow easier search by error code(s) using existing filter API tech. The column is implicitly maintained when the progress JSON changes.

To accommodate this the persisted JSON changed from an array of processes which previously would be wrapped in a Progress object on in the memory model which had the sequence of Processes this root object is now explicit in the stored JSON so it can also contain the errors list. To be backwards compatibly with the JSON array stored previously the extraction pf the progress JSON will auto-wrap any array values in an appropiate object in the SQL so existing data will continue to work.

Other changes

• the ErrorReport of the metadata import now does contain the plain arguments to the message so they can be forwarded to the job progress error list
• the heartbeat job was renamed to housekeeping job
• tracker Error class is now called TrackerImportError in OpenAPI to resolve the name clash with the new generic Error class added
• Metadata import errors are forwarded to the job progress tracking
• cancelling jobs requires either ALL or F_PERFORM_MAINTENANCE authority or must be performed by the user executing the job
• access to job /progress and /errors is restricted to ALL or F_PERFORM_MAINTENANCE authority

Fixes

• sending email messages in case a job failed is now async so any exception in the sending does not cause a rollback of the main TX for transitioning the job to stopped/failed state
• the housekeeping job is re-spawned not only at startup but in each loop cycle that has no single job to execute as the housekeeping is expected to be in that list
• the progress JSON object is now stored as a jsonb object not as string of the escaped JSON (that worked by accident for store and decode)

Automatic Testing

The affected code is covered by many existing test indirectly. No explicit new tests were added yet.

Manual Testing

• open import/export app
• run a metadata import with a file that has errors, for example export metadata, edit the file and change a UID of an object
• check the /api/jobConfigurations/{uid}/errors and /api/jobConfigurations/{uid}/progress endpoints of the job created for the import and check the error added to the file is recorded (find the UID e.g. by the newest METADATA_IMPORT job in the list api/jobConfigurations/gist?order=created:desc)
• make sure the list entries of /errors have a readable message property